Artifacts removal from tissue images

ABSTRACT

The method includes generating, for each of a plurality of original images, a first artificially degraded image by applying a first image-artifact-generation logic on each of the original images; and generating the program logic by training an untrained version of a first machine-learning logic that encodes a first artifacts-removal logic on the original images and their respectively generated first degraded images; and returning the trained first machine-learning logic as the program logic or as a component thereof. The first image-artifact-generation logic is A) an image-acquisition-system-specific image-artifact-generation logic or B) a tissue-staining-artifact-generation logic.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a national phase under 35 U.S.C. § 371 of PCTInternational Application No. PCT/EP2018/075708, which has anInternational filing, date of Sep. 21, 2018, which claims priority toEuropean Patent Application No. 17192764.3, filed Sep. 22, 2017, theentire contents of each of which are hereby incorporated by reference.

FIELD OF THE INVENTION

The invention relates to image analysis, and more particularly to theremoval of artifacts from tissue images.

BACKGROUND AND RELATED ART

In digital pathology, information encoded in a digital image of a tissueslide is extracted from the image for answering various biomedicalquestions, e.g. for assisting a health care professional in thediagnosis and treatment of diseases. The field of digital pathology iscurrently regarded as one of the most promising avenues of diagnosticmedicine in order to achieve even better, faster and cheaper diagnosis,prognosis and prediction of cancer and other important diseases. Digitalpathology techniques are also widely used in the context of drugdevelopment for assisting pathologists in understanding the tumormicroenvironment, learning about patient response, drug mode of actionsand other information available from the tissue images.

The scanned digital tissue images, especially in high magnifications,tend to have several types of noise related to both staining process andscanning process. These noise artifacts present a problem to both manualand automatic analysis. The artifacts can substantially affect and limitboth automatic algorithm results and manual scoring by expertpathologists or at least make such manual or automatic scoring verydifficult and inconsistent.

Various approaches exist for preprocessing tissue images for removing orreducing noise artifacts from the images. For example, Jain et al.(Jain, Viren, and Sebastian Seung. “Natural image denoising withconvolutional networks” Advances in Neural Information ProcessingSystem, 2009) suggested applying convolutional neural networks, Xie etal. (Xie, Junyuan, Linli Xu, and Enhong Chen “Image denoising andinpainting with deep neural networks” Advances in Neural InformationProcessing Systems. 2012) used stacked sparse autoencoders for imagedenoising, Agostenelli et al. (Agostinelli, Forest, Michael R. Anderson,and Honglak Lee. “Adaptive multi-column deep neural networks withapplication to robust image denoising” Advances in Neural InformationProcessing Systems, 2013) used adaptive multi column deep neuralnetworks for image denoising. Other approaches based on wavelets andMarkov random fields have also been described.

US 2017/0213321 A1 describes a computer-implemented method for denoisingimage data. The method includes a computer system receiving an inputimage comprising noisy image data and denoising the input image using adeep multi-scale network comprising a plurality of multi-scale networkssequentially collected. Each respective multi-scale network performs adenoising process which includes dividing the input image into aplurality of image patches and denoising those image patches overmultiple levels of decomposition using a threshold-based denoisingprocess. The threshold-based denoising process denoises each respectiveimage patch using a threshold which is scaled according to an estimationof noise present in the respective image patch. The noising processfurther comprises the assembly of a denoised image by averaging over theimage patches.

RUOHAN GAO ET AL: “On-demand Learning for Deep Image Restoration”, 2017IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), August 2017(2017-08) describes an examination of the weakness of conventional“fixated” models and demonstrates that training general models to handlearbitrary levels of corruption is non-trivial. In addition, an on-demandlearning algorithm for training image restoration models with deepconvolutional neural networks is described. The main idea is to exploita feedback mechanism to self-generate training instances where they areneeded most, thereby learning models that can generalize acrossdifficulty levels.

A problem associated with the current denoising approaches is that thegeneration of the noise-removal logic involves a training phase of amachine learning logic on training data that is hard to obtain in therequired quantity and quality: typically, images have to be annotatedmanually for generating a training data set in which noise artifactsand/or tissue structures which look similar to artifacts but whichshould not be identified as noise artifacts are labeled as “trueartifacts” or “true tissue structures”. This process is highly timeconsuming and requires many hours, days or even weeks of work of one ormore experienced pathologists. Moreover, manual annotation of “realnoise” and “real tissue” may be subjective and different pathologistsmay have different opinions regarding the nature of a particular imagesection that cannot surely be classified as artifact. Thus, the creationof artifact removal program logic currently requires the creation oftraining data sets which is very time consuming and therefore expensive.

Due to the high costs involved with the creation of a training dataset,the training data sets used in praxis are often hardly of sufficientsize to allow for the generation of a noise removal program logic ofsufficient accuracy, because the training of program logic that is ableto accurately remove image artifacts typically requires a large trainingdataset that covers many different manifestations of a particularartifact type.

SUMMARY

It is an objective of the present invention to provide for an improvedmethod of generating program logic configured for removing artifactsfrom digital tissue images and for a corresponding storage medium andimage analysis system as specified in the independent claims.Embodiments of the invention are given in the dependent claims.Embodiments of the present invention can be freely combined with eachother if they are not mutually exclusive.

In one aspect, the invention relates to a digital pathology method. Themethod comprises:

-   -   generating, for each of a plurality of original images        respectively depicting a tissue sample, a first artificially        degraded image by applying a first image-artifact-generation        logic on each of the original images, the first        image-artifact-generation logic being configured for        specifically generating a first type of artifact; and    -   generating a program logic configured for removing artifacts        from digital tissue images. The generation of the program logic        comprises:        -   training an untrained version of a first machine-learning            logic that encodes a first artifacts-removal logic on the            original images and their respectively generated first            degraded images; and        -   returning the trained first machine-learning logic as the            program logic or as a component thereof.

The artifact generation logic can in particular be animage-capture-device specific artifact-generation logic (“option A”) ora staining protocol related artifact (“option B”).

Said features may be advantageous, because the training datasetcomprising “true artifact-free” and “true artifact-containing” imagescan be created fully automatically. Thus, a very large training datasetcan be generated. Typically, high quality images which are basicallyfree of any artifacts or at least free of artifacts of a particular typecan be obtained comparatively easy. For example, artifacts resultingfrom dust in the optical system of a camera can be avoided simply byusing a microscope whose optical system is basically free of dustgrains. This microscope can be used for capturing hundreds and thousandsof images of different tissues stained with different stains accordingto different staining protocols. Thus, a large number of “truenoise-free training images” can be captured easily, and a correspondingnumber of “true noise-containing training images” can be created fromthe former automatically by applying the image-artifact-generationlogic.

Moreover, as the creation of the training images including the degraded,“noisy” training images is performed automatically, the generatedtraining data set may be larger than a manually annotated training dataset, because high-quality, noise free images can typically obtainedeasily. Thus, the accuracy of the trained machine learning logic may behigher.

In a further beneficial aspect, inconsistencies in the annotations inthe training data set inherent to the manual annotation process may beavoided, because the artifacts in the training images are createdautomatically and the artifact generation algorithm “knows” at whichregion in an image an artifact was created.

According to embodiments, the plurality of original images comprisesimages depicting tissues derived from many different tissue types and/ordepicting tissues having been stained with many different stains and/ordepicting tissues having been stained according to many differentstaining protocols and/or depicting tissues whose image was captured bymany different image capturing systems and/or depicting tissues in manydifferent resolutions.

In some embodiments, the automated generation of the degraded version ofan image is performed by a “image degradation” application used forgenerating a training data set that is separate from the program logicused for artifacts removal.

In other embodiments, the automated generation of the degraded versionof an original image is performed by the machine learning algorithm thatis trained to “learn” to remove the artifact. In this case, a machinelearning logic can be used for “learning” one or more artifacts removallogics, whereby only the original images are input into said machinelearning logic during the training as well as during the test phase. Themachine learning logic is configured to operate in two different modes,the training phase mode and the test phase mode. Selectively in thetraining phase mode, this machine learning logic will automaticallygenerate the degraded image versions from the original images receivedas input. In the test phase mode, the trained machine learning logicwill merely remove the artifacts from the original images received asinput. The two different functionalities may be implemented in differentmodules of the machine learning algorithm. For example, the automatedgeneration of the degraded image version can be performed by aparticular layer of a neural network, whereby one or more other layersof said network are used and trained to remove said artifact from animage.

Acquiring high-quality tissue images which are basically free of noiseartifacts from many different tissues and/or from many different imagecapturing devices and/or from tissue sections subjected to manydifferent conditions and then automatically deriving a “degraded”, noisyversion of said image can typically performed much faster and easierthan annotating a training data set having the same broad coveragemanually. Thus, a machine learning algorithm trained to remove artifactsfrom the above described automatically generated data set may be moreaccurate and more robust against variations regarding the tissue type,the stain, the staining protocol and the like because of the increasedsize and diversity of the training data set.

According to embodiments, the training of the untrained version of thefirst machine-learning logic comprises inputting the original images andtheir respectively generated first degraded images into the untrainedversion of the first machine-learning-logic. Each of the first degradedimages is annotated as artifact-afflicted version of a respective one ofthe original images. The first machine learning logic encodes a firstartifacts-removal logic. The training is performed by applying the firstartifacts-removal logic on the first degraded images for generating afirst reconstructed image from each of the first degraded images and byautomatically modifying the first artifacts-removal logic such thatdeviations of the first reconstructed images from the original imagesfrom which the first reconstructed images were derived are minimized.

This “error function minimization” approach for training the artifactsremoval logic is in generality called “supervised learning” machinelearning. Supervised machine learning techniques may be implemented invarious forms, e.g. as a neural network based machine learning logic, assupport vector machine based machine learning logic, or the like.

According to embodiments, the method further comprises generating, foreach of the original images, a second artificially degraded image. Thesecond artificially degraded image is generated by applying a secondimage-artifact-generation logic on each of the original images. Thesecond image-artifact-generation logic is configured for specificallygenerating a second type of artifact. The generation of the programlogic further comprises: training an untrained version of a secondmachine-learning logic that encodes a second artifacts-removal logic onthe original images and their respectively generated second degradedimages; and combining the trained first machine-learning logic with thetrained second machine-learning logic for providing the program logic;the generated program logic is configured for removing artifacts of atleast the first and the second artifact type. Alternatively, the firstand second machine-learning logics can be part of the same machinelearning architecture that is trained in a single training step on atraining data set comprising both the first and second degraded images.

Training one or more machine learning logics on two separate sets ofartificially degraded images may be advantageous in case an artifactremoval logic shall be generated which is capable of selectivelyremoving artifacts of a particular type from an image. For example thegenerated artifacts removal logic may be integrated in an image analysissoftware package that is sold together with or for use in combinationwith a particular microscope. The software may allow the user of thesoftware to select one or more artifacts types, e.g. “blur artifacts”,“focus artifacts”, “dust”, etc. for selectively removing artifacts of aparticular type from an image. As the artifact-removal logics have beentrained separately on different training data sets respectivelyrepresenting the different types of artifacts, the software may allowthe user to selectively select and use e.g. the first artifacts-removallogic or to selectively select and use the second artifacts-removallogic.

In some embodiments, the artifacts removal logic is applied in real timeduring image acquisition, e.g. during an image scanning process. Thus,during the image acquisition process, for each acquired image, a “clean”version is generated by applying one or more artifacts removalalgorithms on the currently acquired image. In other embodiments, one ormore artifacts removal logics are applied on already acquired images,e.g. on images having been acquired in the past and having been storedin the form of an image library on a storage medium.

Thus, in some embodiments, combining the trained first machine-learninglogic with the trained second machine-learning logic for providing theprogram logic may comprise combining two separately addressable andseparately callable trained program logics into the same softwareapplication program. In other embodiments, the combination may be anintegral part of the training process. For example, a single machinelearning software may be trained on a data set comprising the originalimages and a mixture of the first and the second degraded images.

Combination of multiple artifact generation logics and the generationand training of respective artifact removal logics may be advantageousas in reality, combinations of multiple different artifacts arefrequently observed. In some embodiments, the degraded image generatedby one artifacts-generation-logic is input to at least one furtherartifacts generation logic of a different type to generate a two-fold(or, if further artifact-generation-logics are applied, a multi-fold)degraded image. This may allow the generation and training of anartifacts-removal-logic adapted to remove artifacts of many differenttypes from an image in a step-wise manner.

For example, the returned artifacts-removal logic can be configured suchthat it at first removes bad-focus noise such as blur and then removesline noise and speckle noise caused by physical obstructions such ashair, dust, etc. Finally, the artifacts-removal logic can removestaining artifacts such as non specific staining. Accordingly, the firstartificially degraded images could comprise artifacts which simulateblurring artifacts and the second artificially degraded images couldcomprise artifacts which simulate a non-specific staining artifact.Optical noise such as blur does typically not depend on types of noisesuch as the presence of undesired physical objects (e.g. hair, dust) inthe image. However, in some cases, the undesired physical objects maycause an autofocus mechanism to select the wrong focus layer. Moreover,the speckle noise may be affected by the bad-focus noise in case thebad-focus noise results in a blurring of the tissue section as well asof the speckles. Thus, the relationships between artifacts of differenttypes is complex and a challenge for any artifact removal approach.

Due to the time and effort needed for manually annotating image datasets, the size of training data sets comprising annotated tissue imageswith and without artifacts or with combinations of different artifactsis typically small compared to the wide range of combinations ofartifacts that can be observed in reality. Thus, the use of manuallyannotated training data sets inherently bears a strong risk ofover-fitting during training. By computationally generating manydifferent types of artifacts in many different combinations, the risk ofover-fitting may be avoided or at least reduced.

According to embodiments, the second machine learning logic is trainedby inputting the original images and their respectively generated seconddegraded images into an untrained version of the secondmachine-learning-logic, whereby each of the second degraded images isannotated as artifact-afflicted version of a respective one of theoriginal images. The second machine learning logic encodes a secondartifacts-removal logic. The training is performed by applying thesecond artifacts-removal logic on the second degraded images forgenerating a second reconstructed image from each of the second degradedimages and by automatically modifying the second artifacts-removal logicsuch that deviations of said second reconstructed images from theoriginal images from which said second reconstructed images were derivedare minimized. Thus, also the second machine-learning logic and/orany-machine learning logic encoding a multi-step-artifacts-removal logicdescribed further below may be trained such that an error function beingdescriptive of a deviation of the original image and the result ofapplying an image reconstruction logic on a degraded image is minimized.

In some embodiments, the second machine learning algorithm comprises asub-module that automatically generates the artifact-afflicted versionof the original images as described for embodiments above or is asub-module of a machine learning logic that comprises a furthersub-module configured to automatically generate the artifact-afflictedversion of the original images. Thus, the expression “inputting adegraded (artifact afflicted) version of an original image” alsoincludes the option that the degraded version is generated by a softwarelogic that is bundled with the second program logic into a singleexecutable program logic.

In some embodiment, the first, the second and any furthermachine-learning logic for removing artifacts of one or more artifacttypes is implemented in the same machine learning logic, e.g. in thesame instance of a neural network. For example, irrespective of whetherone machine learning logic is trained per artifacts type or whether thesame machine learning logic is trained on training data comprisingartifacts of two or more different artifacts types, neural networks canbe used as the machine learning logic to be trained.

For example, the number of networks that are employed for training therespective artifacts generation logics and artifacts removal logics canbe determined by trial and error. If a network architecture comprising asingle neural network does not perform well, the network architecturemay be extended to two networks or three networks or a different networklayer architecture. Alternatively, multiple different networkarchitectures are trained in parallel and the results generated by thetrained networks on test images are combined and aggregated to generatea consolidated result of multiple networks that has a higher accuracythan the results obtained from any of them separately (“ensemble method”or “ensemble learning”). According to preferred embodiments, the methodcomprises providing a GUI to a user, the GUI enabling the user to selectone or more of a set of predefined artifacts types and a correspondingset of selected artifacts removal logics to be applied automatically onany original image to be processed.

According to embodiments, the program logic configured for removingartifacts that is to be generated is a program logic configured forremoving at least a number N different artifacts types from the digitaltissue images. The number N is an integer larger than 1. The methodfurther comprises: providing, for each of the N different artifactstypes, a respective artifact-generation logic configured forspecifically generating said particular type of artifact; applying,according to a first sequence, the number N differentimage-artifact-generation logics on the original images; thereby, thefirst image-artifact-generation logic within the first sequence takeseach of the original images as input and outputs a degraded version ofsaid image; the degraded image output by the first and any subsequentimage-artifact-generation logic is used as input image by the next oneof the N different image-artifact-generation logics in the firstsequence; thereby, for each of the original images, a first multi-folddegraded image is generated. The generation of the program logic furthercomprises training a machine-learning logic that encodes amulti-step-artifacts-removal logic on at least the original images andtheir respectively generated first multi-fold degraded images; and usingsaid trained multi-step-artifacts-removal logic for providing theprogram logic or for providing a part of the program logic that is to begenerated.

According to some embodiments, the machine-learning logic that encodes amulti-step-artifacts-removal logic is an untrained machine learninglogic that uses the output generated from a plurality of other machinelearning logics for learning to correctly remove multiple differenttypes of image artifacts. The other machine learning logics haverespectively been trained to remove a particular type of image artifact.

In some embodiments, the trained multi-step-artifacts-removal logic iscombined with the trained first machine-learning logic for providing theprogram logic. The resulting program logic is configured forsequentially removing artifacts of each of the N types of artifacts. Theremoval is preferably performed in accordance with the reverse order ofthe first sequence. The training of the machine-learning logic forproviding the trained multi-step-artifacts-removal logic is performedsuch that the multi-step-artifacts-removal logic reconstructs theoriginal image from the multifold-degraded training images. For examplethe multi-step-artifacts-removal logic may be configured to call any ofthe first or second (“single-artifact”)-machine-learning logics indifferent orders and may be configured to learn a sequence of artifactremoval operations that is best suited to reconstruct an image that isvery similar or identical to the original image.

For example, the number N may be “3” and the first sequence may comprisethe following types of artifacts in the following order: a)“overstaining”, b) “dust artifacts” and c) “blurring artifacts”.

Applying multiple artifact-generation logics sequentially on thedegraded image respectively provided by the previously called artifactgeneration logic and training an artifact removal logic such that itreconstructs the original image from the multifold-degraded trainingimages may be advantageous as in praxis, an image may comprise artifactsof many different types, whereby an artifact may be overlaid by,distorted or otherwise affected by a later introduced type of artifact.Thus, the artifacts removal logic may be trained such that it is capableof identifying and compensating also a sequence of image artifacts ofdifferent types and thus may be able to compensate also complexcombinations of artifacts of different types commonly observed in praxisin digital pathology. According to embodiments, the intermediate imagescreated in each degradation step defined by a sequence ofartifact-generation logics are respectively used as “auxiliary loss” inthe network. This means that during training not only the last output ofthe net is compared to the clean image, but in one or more places alongthe processing layers an additional output image (“intermediate image”)is created by the network (during training only) and compared to theoriginal image to create a loss that drives the training of thepreceding layers. For example, a first auxiliary loss image (a firstintermediate image) of the partially degraded image (e.g. withoutblur/defocus) could be computed by applying the blur-artifact-removalalgorithm on the received image, and for each of the subsequentartifacts removal logics in the sequence, an corresponding next“auxiliary loss”/intermediate image is obtained by applying a respectiveone of the artifacts removal algorithms on the intermediate imagegenerated in the previous step until the last loss at the real “deploy”output of the net is the clean image (which should be free of allartifacts belonging to an artifact type of the above mentioned artifactsremoval sequence).

According to some embodiments, the method further comprises applying,according to at least a second sequence, the number N differentimage-artifact-generation logics on the original images. Thereby thefirst image-artifact-generation logic within said second sequence takeseach of the original images as input. The degraded image output by thefirst and any subsequent image-artifact-generation logic is used asinput image by the next one of the different image-artifact-generationlogics in the second sequence. Thus, by executing each of theimage-artifact-generation logics in the second sequence, for each of theoriginal images, a second multi-fold degraded image is created. Thus, inthis embodiment, for each original image, a first multi-fold-degradedimage is obtained by sequentially applying the image-artifact-generationlogics according to the first sequence as described above. In addition,one second multi-fold-degraded image is obtained per original image bysequentially applying the image-artifact-generation logics according tothe second sequence as described above. The first and secondmulti-fold-degraded image may differ from each other, because thesequence of applying the image degradation algorithm typically has animpact on the resulting image. For example, a depicted grain of dust mayhave sharp edges if the dust artifact is added after a blurring artifactwas added to an image and may have blurred edges if the blurringartifact was added after the dust artifact.

According to preferred embodiments, the training of the machine-learninglogic that encodes a multi-step-artifacts-removal logic is performed onat least the original images and their respectively generated first andsecond multi-fold degraded images.

This may be advantageous, because two or more different sequences ofartifacts generation are covered in the training data set and theresulting trained artifacts removal logic will be capable ofcompensating multiple different sequences of generating artifacts in animage. This may be highly useful for compensating complex combinationsof multiple different artifact types where the actual sequence ofartifact generation is unknown or heterogeneous. For example, partialoverstaining artifacts in some image regions may later be blurred as theresult of an out-of-focus artifact. In addition, or alternatively, orselectively in some image regions, the overstaining artifacts may beoverlaid by dust artifacts resulting from dust particles on the lensesof a microscope. So the number and type of artifact combinations mayvary within an image.

Sometimes, the actual physical order of artifact introduction is unclearor unknown.

For example, dirt and debris particles on a tissue section, e.g. hair,cloth fibers or the like, may cause the image acquisition system toselect a wrong focus layer. As a consequence, the dirt may be in-focuswhile all other parts of the image comprise an out-of-focus artifact andappear blurred. In other images, the dirt may not be the cause of anout-of focus error. Rather, an out of focus error may occur due to awrong configuration of the camera system. In this case, the tissue aswell as the dirt particles will show an out-of focus error and both thetissue and the dirt will appear blurred. By training the machinelearning logic not only on different types of artificially createdartifact types, but also un multiple different artifact generationsequences, it may be ensured that the training data set covers the hugecombinatorial space of artifact combinations which are observed inpraxis. It should be noted that manually annotated training data setstypically consist of annotated images which were all derived from a verysmall number of experimental settings and image acquisition systems.Thus, the manually annotated data sets typically do not reflect thecombinatorial complexity and heterogeneity of artifact type combinationsthat is observed in different image acquisition systems using differentstaining protocols, and different tissue types. Thus, manually annotateddata set often resulted in the generation of artifact removal algorithmwhich were not universally applicable for many different types ofmicroscopes, different types of lenses and/or staining protocol. To thecontrary, embodiments of the invention may allow to automaticallygenerate an artifacts removal logic that has been trained on a trainingdata set that covers a huge diversity of artifact types and a hugediversity of sequential artifact combinations. This may be of particularuse e.g. for image analysis software that is designed for a wide rangeof users, e.g. pathologists, molecular biologists, workers in the fieldof quality management and production, etc.

According to embodiments, the method comprises automatically creating aplurality of sequences of the number N differentimage-artifact-generation logics. The plurality of sequences comprisethe first, the second and at least a third sequence by permutating thepositions of the image-artifact-generation logics. The method furthercomprises applying, for each of the plurality of sequences, the number Ndifferent image-artifact-generation logics of said sequence on theoriginal images, whereby the first image-artifact-generation logicwithin in said sequence takes each of the original images as input; thedegraded image output by the first and any subsequentimage-artifact-generation logic in said sequence is used as input imageby the next one of the different image-artifact-generation logics insaid sequence, thereby generating, for each of the original images andfor each of the plurality of sequences, a multi-fold degraded image. Thetraining of the untrained version of the machine-learning logic thatencodes a multi-step-artifacts-removal logic is performed on at leastthe original images, the plurality of sequences and their respectivelygenerated multi-fold degraded images.

The automated sequence generation may be advantageous, because it allowsto cover a huge combinatorial space of artifact type combinations. Forexample, the number N may be “3” and the set of artifact types to becovered may comprise artifact types AF1, AF2 and AF3. The permutation ofthe three artifact types will result in an automated generation of thefollowing sequences S1 to S6:

-   -   S1: AF1, AF2, AF3    -   S2: AF1, AF3, AF2    -   S3: AF2, AF1, AF3    -   S4: AF2, AF3, AF1    -   S5: AF3, AF1, AF2    -   S6: AF3, AF2, AF1

By applying each of the above 6 artifact generation logics on aparticular input image I, 6 multi-fold degraded images will be generatedfor said image. In addition, in some embodiments, also three single-folddegraded images may be generated and fed into the machine learninglogic, whereby each of the single-fold-degraded image may be generatedby applying either AF1, or AF2 or AF3 on the original image.

So for 100 original images and for 3 different types of artifacts, 6sequences S1-S6, 600 multi-fold degraded images and three single-folddegraded images will be created from each original image and will be fedinto the machine learning algorithm.

In some embodiments, the permutation of the artifact types comprisesgenerating sequences which comprise only a sub-set of the availableartifact-generation-logics. In the depicted example, those additionalsequences would comprise:

-   -   S7: AF1, AF2    -   S8: AF1, AF3    -   S9: AF2, AF1,    -   S10: AF2, AF3    -   S11: AF3, AF1    -   S12: AF3, AF2

So for 100 original images and for 3 different types of artifacts, 12sequences 51-S12, 1200 multi-fold degraded images and three single-folddegraded images will be created from each original image and will be fedinto the machine learning algorithm.

As can be inferred from the above depicted examples, the combinatorialspace is huge. Thus, embodiments of the invention may allow generating atraining data set that covers much more artifact combinations than anymanually annotated data set could cover.

In some embodiments, the length of the generated sequences is limited toa chain length of a predefined maximum size, e.g. two, three or fourelements. This may increase performance.

In addition, or alternatively, only a single sequence or a few (lessthan four sequences) of applying artifact-generation-logics are definedmanually. This single sequence (or each of the few sequences) in thiscase represent the typical sequence of artifact generation in a tissuestaining and image acquisition workflow. This embodiment may increasethe performance by reducing the combinatorial space and may be suitedfor application scenarios where the typical sequence of artifactgeneration is known in advance.

According to some embodiments, a GUI is provided that enables a user toselect one or more different types of artifact removal logics, therebyenabling the user to manually specify a sequence of artifacts removallogics to be applied on an input image. According to preferredembodiments, however, the sequence of artifact-removal logics to be usedby the trained multiple-artifacts-removal logic is learned in a machinelearning process and is encoded in the trained multiple artifact-removallogic.

According to embodiments, the first and/or second machine-learning logicand/or the machine-learning logic that encodes themulti-step-artifacts-removal logic is a neural network. In someembodiments the first and/or second machine-learning logic and/or themulti-artifact machine-learning logic belong to the same single neuralnetwork. In other embodiments, the first and/or second machine-learninglogic and/or the multi-artifact machine-learning logic are implementedin a separate network, respectively, and the networks are combined in a“super” neural network or in another form of software logic that isadapted to integrate the results provided by the individual networks.For example, the “super” neural network could be trained toautomatically combine the output of the individual networks such thatmany different types of artifacts are removed and whereby the “super”network coordinates the operation of the individual network for removingdifferent types of artifacts.

According to embodiments, the first and/or second machine-learning logicand/or the machine-learning logic that encodes themulti-step-artifacts-removal logic is an autoencoder.

According to embodiments, the first and/or second artifact is selectedfrom a group comprising: staining artifact; scanning artifact;tissue-fold artifact; line-artifact (e.g. air bubbles, fibers).

Typically, when feeding the machine-learning logic with training datacomprising the original images and the degraded versions of the imageswhich selectively comprise the artifacts of one of the artifact types,the artifacts-removal logic generated for each of said artifact typesrespectively is configured for selectively removing artifacts of saidsingle type.

According to embodiments, the scanning artifact is selected from a groupcomprising bad-focus-artifacts (e.g. blurred images resulting e.g. froman erroneous selection of the focus layer or from a misconfiguration ofa microscope), stitching-artifacts (e.g. effects and patterns resultingfrom scanner-stitching) and speckle-noise-artifacts (e.g. dust in theoptics of the image acquisition system, bad pixels caused by defects inthe sensor hardware of the camera).

According to embodiments, the staining artifact is selected from a groupcomprising background-staining artifact, non-specific staining artifact,residual staining artifact, anthracotic pigment artifact.

According to embodiments, the first and/or secondimage-artifact-generation logic is configured to specifically generatebad-focus-artifacts by applying a point spread function (PSF) or anapproximation, e.g. a Gaussian approximation, of the point spreadfunction on each of the original images.

For example, the PSF and/or the Gaussian approximation (“Gaussianblurring”) is applied locally on the image. For example, all pixel blobshaving an intensity above a predefined threshold can be identified andthe PSF can be applied on each of the identified pixel blobs forgenerating a degraded image comprising “blurred” blobs which simulatethe blurring generated by the selection of a wrong focus layer.

According to embodiments, the PSF is generated by mounting micro-beadsof a known size on an empty tissue slide or on a tissue slide togetherwith or in addition to the tissue sample; for example, the micro-beadsize can be chosen such that the expected size of the micro-beaddepicted in a digital image of the tissue slide is one pixel or adefined set of pixels. The expected size of the depicted bead willdepend on the size of the bead, the magnification, resolution and/orother properties of the image acquisition system; then, the imageacquisition system captures a digital image of the slide with the tissuesample and the micro-beads; the obtained digital image is referred to inthe following as “calibration image”; an image analysis system analyzesthe obtained calibration image for measuring the size of the image areathat actually depicts the bead; the image analysis system also acquiresthe pixel pattern observable within the image area that actually depictsthe bead and fits a mathematical model to that pattern for generating aPSF; the PSF is configured to generate the extracted pattern if appliedon an image blob having the size of the expected bead-area.

The generated PSF is then integrated into a machine-executable code,e.g. a Java or C program code, for providing animage-artifact-generation logic adapted to generate speckle noiseartifacts which reproduce the speckle noise generated by a particularimage acquisition system or image acquisition system type. The PSF ofthe image-artifact-generation logic can then be applied on eachhigh-intensity blob of an original image for creating a degraded imagethat simulates a bad focus artifact for each or at least some of thehigh intensity blobs of the original image. A “high intensity blob” asused herein is a region of adjacent pixels whose intensity issignificantly higher than the intensity of the surrounding pixel area.For example, the high intensity blobs can be identified by applying anintensity threshold or similar image processing and segmentationoperations. Instead of the PSF, a Gaussian filter model could likewisebe created by fitting mathematical functions to an image deteriorationgenerated by a particular image acquisition system. A Gaussian filtercan be considered as a “simple” approximation for the PSF and is beapplied in a similar manner.

This may be advantageous as bad-focus errors frequently occur in digitalpathology and the PSF based artifact-generation logic has been observedto faithfully reproduce this type of artifact.

According to embodiments, the method comprises generating the PSF or theGaussian filter empirically as described above for each of a pluralityof different image acquisition systems (e.g. microscopes or slidescanners) or image acquisition system types (e.g. microscopes or slidescanners of a particular device type). Each of the PSFs (or Gaussianfilters) is integrated into a respective artifact-generation-logic,which is in the following referred to as “image acquisition systemspecific artifact-generation-logic, IASS-artifact-generation-logic, orimage acquisition type specific artifact generationlogic—IATSS-artifact-generation-logic.

According to embodiments of the invention, the method for generating anartifact removal logic is performed for each of a plurality of differentimage acquisition systems (IASs). This means that from each of the IASs,a set of calibration images is obtained and a PSFs (or Gaussian filter)is obtained for each of the IASs by fitting a mathematical function suchthat the deviation of the measured shape and size of the beads to theexpected size and shape of the beads is described (modeled) by thefitted function. The generated PSF (or Gaussian filter) is integrated inan artifact-generation-logic, thereby generating an IASSartifact-generation-logic adapted to simulate (bad-focus) noise artifactspecifically generated by each of the plurality of IASs respectively. Byapplying the IASS artifact-generation-logic on original images or onimages having been degraded by one or more otherartifact-generation-logics, it is possible to generate a trainingdataset that simulates the out of focus errors generated by a particularIAS. Moreover, these “simulated” out-of-focus errors may be combinedwith one or more other artifact types, thereby generating a set oftraining images that faithfully reproduce the spectrum of artifacts thatmay be obtained by using a particular microscope.

According to other embodiments, the method for generating an artifactremoval logic is performed for each of a plurality of different IAStypes. The method is performed like the method for generating anartifact removal logic for each of a plurality of different imageacquisition systems (IASs) described above, with the difference thatmultiple IAS of the same type are used for generating a pool ofcalibration images and that a single PSF (or Gaussian filter) for allIAS of the same type, whereby the single PSF (or Gaussian filter) isgenerated such that it simulates (bad-focus) an averaged noise artifactgenerated by all IAS of the same IAS type.

As a result, an image artifact removal logic is generated for each ofthe IAS types that is capable of compensating the bad-focus artifactsgenerated specifically by this particular IAS type. This may beparticularly advantageous for manufacturers of microscope systems, slidescanners or other forms of image acquisition systems as the artifactsgenerated by a particular IAS type may be faithfully reproduced and maybe used for automatically generating noise-removal logic that is capableof compensating bad focus artifacts generated by a particular type ofIAS.

According to embodiments, the first and/or secondimage-artifact-generation logic is configured to specifically generatetissue-fold-artifacts in each of the original images, the generation ofa tissue-fold-artifact comprising:

-   -   generating at least two image parts of the original image by        cutting the original image across a randomly selected line;    -   overlaying the at least two image parts along the line; and    -   merging the overlap, e.g. using alpha blending.

According to embodiments, the first and/or secondimage-artifact-generation logic is configured to specifically generatespeckle-noise-artifacts by applying a salt-and-pepper noise function oneach of the original images.

For example, applying a salt-and-pepper noise function on an input imagecan comprise:

-   -   generating, for each pixel in the input image, a random value,        e.g. between 0 and 1;    -   specifying a salt-and-pepper noise threshold (e.g. 0.1 for a        probability of 10% for a pixel to be affected by the        speckle-noise);    -   determining, for each pixel in the input image, if the random        value generated for the pixel is smaller than the        salt-and-pepper noise threshold;    -   if true (the random value is smaller than the threshold), the        pixel is replaced by a black or white pixel; in some        embodiments, the replacement step is performed as an obligatory        step, in other embodiments, the replacement step is performed        only at a predetermined probability, e.g. 30%.

After a pixel was replaced by a black or white pixel in the previousstep, the pixel is in some example implementations grown to a biggersize; for example, the pixel can be extended by randomly selecting aradius within a predefined radius range, e.g. 1-5 pixels, and settingall pixels within the radius around the replaced pixel to the sameintensity value (black or white) as the intensity value of thereplacement pixel.

Thus, black or white speckles of variable sizes can be obtained. Thespeckles represent dust speckles, bad pixels (as generated by defect CCDelements of a camera) and other forms of impulse noise sometimes seen onimages. This noise is typically characterized by sharp and suddendisturbances in the image signal that presents itself as sparselyoccurring white and black pixels. While state of the art approaches usedmedian filters or morphology filters for noise removal, this approach isbased on using a function that simulates the noise and training amachine learning logic on original images and speckle-noise images forautomatically removing speckle-noise-artifacts. It has been observedthat this approach is often more accurate and is more generic. Inparticular, as the noise is simulated, large training data sets can begenerated which cover any possible combination of different noiseartifact types, whereby one or even more chronological sequences ofnoise artifact generation may be covered. For example, noise specklesresulting from dust in the camera optics may not be affected byout-of-focus errors while noise speckles resulting from dust on thetissue slide will be affected from out-of-focus errors.

According to embodiments, the first and/or secondimage-artifact-generation logic is configured to specifically generateline-artifacts. A “line artifact” as used herein is an image artifactthat has the shape of a curved or straight line. For example, lineartifacts can be caused by cloth fibers or the outline of gas bubblesunder the coverslip of a tissue slide. The generation of theline-artifacts comprises generating a plurality of polynomial curvesusing randomly generated values as curve parameters; and inserting blacklines along the coordinates of the polynomial curves in each of theoriginal images. Preferably, the thickness of the inserted linesrepresents the typical line thickness of physical objects expected tocause the line artifacts in a digital image captured by an IAS like amicroscope or a slide scanner under a given magnification. Said physicalobjects can be, for example, cloth fibers, hair and/or gas bubbleoutlines. According to embodiments, the image-artifact-generation logicis configured to specifically generate line-artifacts whose length,thickness and/or curvature represents different types of physicalobjects, e.g. hair, gas bubble outlines and/or cloth fibers.

For example, the typical width of a hair is between 0.05 to 0.1 mm.Given a magnification of the IAS of 10×, the line thickness of theinserted line is in the range of 50-100 pixels for simulating“hair-related” line artifacts. In some examples, the polygon has acurvature degree of 0-3°. It has been observed that, at least whengenerating degraded training images with limited size (e.g. 512×512pixels), a small curvature is sufficient to faithfully reproduce thecurvature of most types of fiber-shaped artifacts.

According to embodiments, the first and/or secondimage-artifact-generation logic is configured to specifically generatestitching artifacts. The generation of the stitching artifactscomprises: scanning an empty slide using an IAS (in particular a slidescanner) whose stitching artifacts shall be removed; automaticallyanalyzing the scanned image of the empty slide for generating a squarefilter that, when applied on an image with homogeneous intensitydistribution, generates the measured stitching elements; generating astitching-artifact-generation-logic by integrating the square filter inan executable program logic. The generatedstitching-artifact-generation-logic can then be used as the first orsecond artifact-generation-logic for generating stitching artifactstypically produced by the image acquisition system or by a particulartype of image acquisition system.

According to other embodiments, a two-dimensional function is definedad-hoc to simulate the typical kind of stitching artifacts which in manycases result from a non-homogenous intensity distribution across thefield of view of a specific tile. This two-dimensional function is thenapplied to the original image of the empty slide to generate anon-homogenous mean intensity along the original image. For example, theapplication of the function for generating a new, degraded image with astitching artifact can be performed in accordance with the followingformula:Z=[C+0.5*(X/a−floor(X/a)+Y/b−floor(Y/b))]/[1+C];new_image=original_image*Z.

Thereby, X is the image width coordinate having a value between 0 andimage_width; Y is the image height coordinate having a value between 0and image_height; a is the desired (predefined) tile artifact width; bis the desired (predefined) tile artifact height; Z is the resulting“tile artifact factor image” consisting of a grid of tiles with width aand height b; C is a “minimum darkness factor” chosen such that C/(1+C)is the lowest quantity by which an original image pixels is multiplied.The parameter orig_image is the original untiled image and “new_image”is the output image with the tiling artifact.

For example, the factor C can be chosen such that at the right lowercorner of each grid element a factor of 0.1 (C is 1/9) is applied to thepixel intensities of the original image and at the left upper corner ofeach grid element, no additional intensity is applied to the pixelintensities of the original image (factor 1).

If C would be chosen to be “1”, then the pixel of the top-left corner ofeach grid element would be 0.5*original pixel value. If C would bechosen to be “0” than the top-left pixel of each grid element would be0. In other embodiments, the XY-axes definition may differ and thegradient produced by the above mentioned formula goes in oppositedirection.

For example, a and be can be selected according to specifications of theimage acquisition system used. For a 20X scan with the Ventana HTScansystem, the parameter a may have the value 1658 and the parameter b mayhave the value 1152. An artificially degraded image comprising stitchingartifacts generated in accordance with the above function is depicted inFIG. 9 .

The method comprises applying the square filter on each of the originalimages for respectively obtaining the degraded version of the originalimage. The degraded version of the original image (or any other form ofinput image provided) will show the stitching artifact generated by thisparticular IAS.

For example, the application of the square filter on an input imagecomprises virtually dividing the image to squares of a width and heightequal to the observed stitching artifact width and height and thenmultiplying the intensity values of the pixels of the stitching artifactsquare with the pixel intensities of each the squares of the input imageonto which the square filter is mapped and applied. Thereby, a patternof corrupted, “degraded” squares is generated that simulates thestitching effect of the IAS or IAS type for which the square filter wasgenerated.

According to embodiments, the first and/or secondimage-artifact-generation logic is configured to specifically generatenon-specific-staining artifacts. The generation of thenon-specific-staining artifacts comprises performing an operationselected from a group comprising:

-   -   generating a color-noise function, the color noise function        being a color-specific salt-and-pepper function or an empirical        heuristics derived from images of tissue slides with regions        with residual stain;    -   applying the color-noise function on each of the original        images, thereby generating, for each of the original images, a        degraded image with non-specific-staining artifacts.

Automatically generating non-specific staining artifacts for generatinga training data set with degraded, “virtually”, non-specifically stainedtraining images and training an artifacts removal algorithm on saidtraining data set may be advantageous, as non-specific stainingartifacts are complex noise patterns that often represent obstacles furfurther image analysis. Non-specific staining artifacts are artifactscreated by staining dyes that were not washed out during the stainingprocess. The remaining dyes create patches of various sizes with aspecific color of the dye. The colors used for this error are collectedfrom the images that need to be denoised according to the dyes used inthe tissue staining process

For example, the application of a color-noise function having the formof a color-specific salt-and-pepper function noise function on an inputimage comprises:

-   -   generating, for each pixel in the input image, a random value,        e.g. between 0 and 1;    -   specifying a salt-and-pepper noise threshold (e.g. 0.1 for a        probability of 10% for a pixel to be affected by the        non-specific-staining noise);    -   determining, for each pixel in the input image, if the random        value generated for the pixel is smaller than the        salt-and-pepper noise threshold;    -   if true (the random value is smaller than the threshold), the        pixel is replaced by a pixel of the specific color of the        color-specific salt-and-pepper function;    -   this color is typically the color of the for which a respective        image artifact training data set is to be created and for whose        non-specific staining artifacts a respective stain specific        artifacts-removal logic shall be created; in some embodiments,        the replacement step is performed as an obligatory step, in        other embodiments, the replacement step is performed only at a        predetermined probability, e.g. 30%.

According to other examples, a color-noise function which is based onthe “Perlin noise generator” approach is generated and used forgenerating degraded image versions comprising the non-specific-stainingnoise. Perlin noise is a type of gradient noise developed by Ken Perlin(SIGGRAPH paper “An image Synthesizer”, 1985. The Perlin noise generatoris a technique used to produce natural appearing textures on computergenerated surfaces for motion picture visual effects. The development ofPerlin Noise has allowed computer graphics artists to better representthe complexity of natural phenomena in visual effects for the motionpicture industry. It has been surprisingly observed that Perlin noisealso can be used for faithfully reproducing unspecific stainingartifacts of certain dyes.

According to embodiments, the first and/or secondimage-artifact-generation logic is configured to specifically generatehue shift artifacts that are generated by a particular image acquisitionsystem, the method further comprising:

-   -   taking, by the image acquisition system, a digital image of an        empty slide;    -   analyzing the digital image of the empty slide for automatically        generating a hue shift filter that is configured to simulate the        hue-shift artifact generated by the image acquisition system; of        course, this and the previous step may also be applied on many        digital images of an empty slide to improve the data basis used        for generating the hue shift filter; and    -   applying the hue shift filter on an input image, thereby        generating a degraded version of the input image.

This may be advantageous, as IAS-device specific chromatic aberrationsand/or intensity shifts and/or device-specific blur effects can becorrected in a piece of software logic. For example, the softwareprogram typically provided in combination with an image acquisitionsystem may comprise the stitching filter and the hue shift filter whichare respectively adapted to correct the specific hue shift and stitchingerror created by the hardware components of this particular IAS.

According to embodiments, the image-artifact-generation logic is adaptedto create image-capturing-device-specific artifacts. The methodcomprises providing a plurality of image acquisition systems; for eachof the plurality of image acquisition systems: creating the firstimage-artifact-generation logic according to any one of the embodimentsand examples described herein, whereby the firstimage-artifact-generation logic is specific to said image acquisitionsystem; capturing, by said image acquisition system, a plurality oforiginal images of a tissue sample; generating, for each of theplurality of original images a respective first artificially degradedimage by applying the created image-acquisition-specific firstimage-artifact-generation logic on each of the original images;generating an image-acquisition-system-specific program logic configuredfor removing artifacts from digital tissue images captured by said imageacquisition system, the generation comprising training an untrainedversion of a first machine-learning logic that encodes a firstartifacts-removal logic on the original images and their respectivelygenerated first degraded images; and returning the trained firstmachine-learning logic as the image-acquisition-system-specific programlogic or as a component thereof.

Said features may be advantageous in particular in the context of imageanalysis in the biomedical domain, because often, image acquisitionsystems are highly complex systems whose components have been assembledspecifically for the needs and demands of a particular research group orlaboratory. For example, the components of a complex microscope systemor slide scanner system may be obtained from different manufacturers,some components may be newer than others due to an exchange ofdysfunctional or outdated parts, and the combination of components maybe the result of an extensive and complex consultation between arepresentative of microscopy systems and the respective working group.In many cases, in particular in the context of biomedical research, thecombination of components of a complex image acquisition system isunique. Hence, the automated creation of a IAS-specific artifactgeneration and removal algorithm may greatly increase the accuracy ofall further image analysis steps performed on the images acquired bysaid IAS.

In a further aspect, the invention relates to a digital pathologyimage-correction method for digital images depicting a biologicalsample. The method comprises: receiving a digital image of thebiological sample, the digital image comprising an artifact; applyingthe program logic generated in accordance with the method for generatingthe artifacts-removal-program logic described herein for embodiments ofthe invention on the received digital image for generating anartifact-corrected image; and returning the artifact-corrected image.

In a further aspect, the invention relates to a computer programcomprising computer-interpretable instructions which, when executed by aprocessor, cause the processor to perform a method according to any oneof the embodiments described herein. For example, the computer programcan be stored on a digital volatile or non-volatile storage medium orcan be stored in a cloud environment and be provided via a network toone or more clients.

In a further aspect, the invention relates to an image analysis systemcomprising a storage medium and a processor. The storage mediumcomprises a plurality of original images, each original image depictinga tissue. The processor is configured for generating a program logicconfigured for removing artifacts from digital tissue images. Theprocessor is configured to perform a method comprising: generating, foreach of the original images, a first artificially degraded image byapplying a first image-artifact-generation logic on each of the originalimages, the first image-artifact-generation logic being configured forspecifically generating a first type of artifact; The processor being inaddition configured for generating the program logic, the generationcomprising training an untrained version of a first machine-learninglogic that encodes a first artifacts-removal logic on the originalimages and their respectively generated first degraded images; andreturning the trained first machine-learning logic as the program logicor as a component thereof.

Any of the artifact-generation logics described herein can be applied onan original image for obtaining a single-fold degraded image. Inaddition, they can be applied on an already single-fold or multi-folddegraded image output by one or more other artifact-generation logic. Inthis case, the input image used by the artifact-generation logic is notan “original image” but a degraded version of an original image whichcan also be referred to as “intermediate degraded image” or“intermediate image”.

An “original image” as used herein is an image that is acquired by animage capturing system or a derivative thereof. For example, an originalimage can be an RGB image or grey scale image obtained from a brightfield microscope or can be a grey scale image obtained from a particularchannel of a fluorescence microscope. It can also be a multi-channelimage that is then decomposed into single-channel images. Preferably, anoriginal image is an image having at least standard image quality, andpreferably a good quality. Preferably, the original image should bebasically free of artifacts, in particular be basically free ofartifacts which shall be introduced later by applying anartifact-generation-logic on the original image.

An “autoencoder”, “autoassociator” or “Diabolo network” as used hereinis a network architecture for which the input and output data sizes areequal. For example, the input image may be an RGB image of a dimensionof 100×100 pixels and the output generated by the autoencoder for thisimage can be also an RGB image of a size of 100×100 pixels. Anautoencoder is adapted to learn a function from R^(N)=>R^(N) where N isthe number of pixels in the input and output images, respectively. Theautoencoder can be an artificial neural network used for unsupervisedlearning. An autoencoder is configured to learn a representation(encoding) for a set of data, typically for the purpose ofdimensionality reduction. In particular, an autoencoder can beconfigured for learning generative models of data by learning anapproximation to identity function using backpropagation.Architecturally, the simplest form of an autoencoder is a feedforward,non-recurrent neural network very similar to the multilayer perceptron(MLP)—having an input layer, an output layer and one or more hiddenlayers connecting them —, but with the output layer having the samenumber of nodes as the input layer, and with the purpose ofreconstructing its own inputs (instead of predicting the target value Ygiven inputs X). Therefore, autoencoders are unsupervised learningmodels. An autoencoder can be, in particular, a denoising autoencoder.

An “image artifact” or “artifact” as used herein is an optical featureor pattern that appears in an image of a tissue, in particular of astained tissue, and is the result of an error or undesired event thatoccurred during the preparation, fixation or staining of a tissue sampleor during image acquisition of a digital image of the tissue sample.

A “staining artifact” as used herein is an artifact which is caused byan error or undesired event during the sample preparation, fixation orstaining. A staining artifact can result from a number of causesincluding improper fixation, improper type of fixative, poordehydration, improper reagents, poor microtome sectioning, improper typeor concentration of stain, improper staining temperature, improperstaining duration, improper pH of the staining solution and the like.

A “scanning artifact” as used herein is an artifact which is caused byan error or undesired event during the image capturing process.

A “tissue-fold artifact” as used herein is an artifact that results froman erroneous generation of a tissue sample or from an error thatoccurred while mounting the tissue sample on a slide, whereby thedepicted tissue comprises one or more folds. For example, in case atissue is insufficiently dehydrated prior to clearing and infiltrationwith paraffin wax, the tissue may not be sectioned correctly on themicrotome, leading to tearing and holes in the sections. A “tissue foldartifact” can also be produced when tissue adheres to the under surfaceof the blade. This type of artifact is commonly observed when the bladeused for cutting the tissue is dull and/or when the tissue is fattytissue.

A “line-artifact” as used herein is an artifact that has the shape of acurved or straight line or comprises sections having the shape of acurved or straight line. For example, air bubbles in the tissue sample(in particular the outline of the bubbles), hair and fibers of cloth orof other origin often have the form of a line, e.g. the form of acircular line in the case of air bubbles. Bubbles under the coverslipmay form when the medium used for mounting the tissue sample on a slideis too thin, because, as the medium dries, air is pulled in under thecoverslip. Contamination of clearing agents or coverslipping media mayalso produce a bubbled appearance under the microscope.

An “anthracotic pigment artifact” as used herein is an image region thaterroneously appears to be strongly stained because of minute carbonparticles deposited in lung tissue that give the tissue the blackcoloration seen in anthracosis.

A “non-specific staining artifact” as used herein is an image artifactcreated by staining dyes that were not washed out during the stainingprocess. Thus, a non-specific staining artifact represents atissue-region or non-tissue region that comprises a significant amountof stain that was supposed to selectively stain specific tissue regionsbut was observed to stain additional regions inside of or outside of thetissue. The remaining dyes create patches of various sizes with aspecific color of the dye which typically do not comprise anytissue-specific patterns or intensity gradients. Some forms of the“non-specific staining artifacts” may also be referred to as “residualstaining artifacts”.

A “hue shift artifact” as used herein is an image artifact in which theintensity values of all pixels of an image or of some regions of animage deviate from the expected pixel intensities. For example, a hueshift artifact can be generated by some erroneous hardware of an IASthat comprises a sensor which is more sensitive to light on the leftside than on its right side. Such reproducible, systematic errors of thehardware IAS may be compensated by loftware logic according toembodiments of the invention. A “Point spread function (PSF)” as usedherein is a function that describes the response of an imaging system,e.g. a microscope, to a point source or point object.

A “filter” as used herein is a function of a program logic thatspecifies one or more image processing operations. For example, a filtermay comprise a function that increases or decreases the pixel intensityvalues in dependence on the x- and y coordinate of the pixel in an inputimage.

A “logic” as used herein is a set of machine-interpretable instructionsthat, when executed by a processor, causes the processor to perform atleast one function, e.g. the removal of a particular artifact from aninput image.

A “digital pathology method” as used herein is a method to be used inthe field of digital pathology. Digital pathology is an image-basedinformation environment which is enabled by computer technology thatallows for the management of information generated from a digital slide.Digital pathology is enabled in part by virtual microscopy, which is thepractice of converting glass slides into digital slides that can beviewed, managed, and analyzed on a computer monitor. The field ofdigital pathology is currently regarded as one of the most promisingavenues of diagnostic medicine in order to achieve even better, fasterand cheaper diagnosis, prognosis and prediction of cancer and otherimportant diseases.

BRIEF DESCRIPTION OF THE DRAWINGS

The patent or application file contains at least one drawing executed incolor. Copies of this patent or patent application publication withcolor drawing(s) will be provided by the Office upon request and paymentof the necessary fee. These and other aspects will now be described byway of example with reference to the accompanying drawings, of which:

In the following embodiments of the invention are explained in greaterdetail, by way of example only, making reference to the drawings inwhich:

FIG. 1 is a flowchart of a method of generating an artifacts-removallogic;

FIG. 2 is a block diagram of an image analysis system;

FIG. 3 is a digital tissue image with a tissue fold artifact;

FIG. 4 is a digital tissue image with a speckle noise artifact;

FIG. 5 is a digital tissue image with a line artifact;

FIG. 6 is a digital tissue image with a stitching artifact;

FIG. 7 is a digital tissue image with a non-specific staining artifact;

FIG. 8 is a digital tissue image with a bad-focus artifact;

FIG. 9 is a digital tissue image with a stitching artifact created withimage acquisition system specific parameters;

FIG. 10 is a block diagram of a “naïve” Neural Network architecture;

FIG. 11 is a block diagram of an “auxiliary loss” Neural Networkarchitecture;

FIG. 12 a is a block diagram of a Basic encoder Unit;

FIG. 12 b is a block diagram of a Residual Encoder Unit; and

FIG. 12 c is a block diagram of a Basic Decoder Unit.

FIG. 1 is a flowchart of a method of generating an artifacts-removallogic according to one exemplary embodiment of the invention. The methodcan be performed, for example, by an image processing system accordingto a further embodiment of the invention that is depicted in FIG. 2 . Inthe following, the method of FIG. 1 will be described by makingreference to FIG. 2 .

FIG. 2 shows an image analysis system 200. The system comprises a dataprocessing system 202 with one or more processors 228, e.g. a standarddesktop computer system, a notebook, a tablet computer or a servercomputer system. The image analysis system 200 comprises or isoperatively coupled to an image acquisition system 204, e.g. abrightfield or fluorescence microscope or a slide scanner. In the imageacquisition system comprises a camera 212. The data processing system202 comprises an interface for receiving the digital images of tissueslides captured by camera 212. For example, the interface can be anetwork connection, an USB stick, a CD-ROM drive or the like. Theprocessing system 202 comprises a non-volatile storage medium 240 whichstores a plurality of original images 208. The original images can beRGB images or grayscale images as generated by the camera 212 are can bepreprocessed images whose resolution, contrast, color space or otherfeature may differ from the image that was originally captured by thecamera. The following, it will be assumed that the original images 208are rgb images captured by a camera of a brightfield microscope.

The storage medium may further comprise a plurality of artifactgeneration logics 230-236 which can respectively be implemented asstandalone software applications. Alternatively, the artifact generationlogics 230-236 are implemented as components of a single softwareapplication. Preferably, a user is enabled to specifically select one ormore of the artifact generation logics for generating degraded imagescomprising a single specific type of artifact or comprising two or moreartifacts which were generated according to one or more predefinedchronological sequences of artifact generation. FIG. 2 shows only asubset of the artifact generation logics described herein: astitching-artifact-generation-logic 230, aline-artifact-generation-logic 232, a speckle-artifact-generation-logic234, and an unspecific-staining-artifact-generation-logic 236. Furtherartifact generation logics or a different subset of artifact generationlogics described herein may be comprised in the data processing systemsof other embodiments. Depending on the embodiment and/or on the type ofartifact, the respective artifact-generation-logic can be specifiedmanually or automatically based on a machine learning approach on tissueimages comprising the “real” artifact. Depending on the embodimentand/or the type of artifact, the artifact-generation-logic can be inspecified once for a particular type of artifact, or can be specifiedonce for a particular type of image acquisition system 204, or can bespecified once for an individual image acquisition system. For example,the stitching-artifact-generation-logic 230 can be obtained specificallyfor a particular type of microscope or even for an individual microscopesystem by acquiring calibration images via a particular imageacquisition system and fitting a square filter or other mathematicalfunction such that the filter faithfully reproduces the stitchingpattern observed within the calibration images (not shown).

The larger the number of different artifact-generation-logics stored inthe storage medium 240, the larger the types of artifacts that can begenerated automatically, the larger the number of combinations ofartifacts of different types and the larger the training data set thatcan be generated automatically from the original images 208. Preferably,the original images 208 are high quality images which are basically freeof image artifacts, or at least are basically free of image artifacts ofan artifact type that shall be artificially generated by one of theartifact-generation-logics in the following steps.

In a first step 102, the image analysis system applies one of theimage-artifact-generation-logics 230, 232, 234, 236 on each of theoriginal images 208. Thereby, for each of the original images, anartificially degraded image is generated. For example, if thestitching-artifact-generation-logic 230 is applied on an original image,the resulting degraded image will show a stitching pattern that isadded, superimposed, multiplied or otherwise combined by theartifact-generation-logic 230 with the pixels of the original image.Depending on the embodiment, the generation of a degraded version of anoriginally received image can be performed right after or during theimage acquisition by an image acquisition system or can be performedmany days after the images were acquired.

After having created, for each of the original images, a respectivedegraded image by applying one of the artifact-generation-logics230-236, a training data set is obtained that comprises the originalimages 208 and the respectively (single-fold) degraded images 210.A.

Then, an artifacts removal logic 218 is generated. The generation of theartifacts-removal-logic 218 comprises a step 104 of training a machinelearning logic that encodes an artifacts-removal logic on the originalimages 208 and the degraded images 210 and returning, in step 106, thetrained machine-learning logic 218.

For example, in case the training data set consists of original imagesand degraded images generated by the stitching-artifact-generationlogics 230 (and is free of images comprising any other artifact type),the generated artifacts removal logic 218 will basically be configuredfor selectively removing, via the stitching-artifact-removal-logic 220,stitching artifacts of a particular camera or camera type. The artifactsremoval logic will not be able to remove line artifacts, speckleartifacts or the like. In case the training data set 210 in additioncomprises degraded images generated by other artifact generation logics232-236, the artifacts removal logic 218 will be trained on aninformation-enriched training data set and will be adapted to removeartifacts of multiple different types. Preferably, the training data setthat is used for generating the respective artifact-removal logics220-226 comprises meta-data being descriptive of the particular artifactgeneration logic that generated a particular degraded image from aparticular one of the original images 208.

In some embodiments, the artifacts-removal logic 218 is configured tosolely and selectively remove artifacts of a particular, single artifacttype from an image. According to the embodiment depicted in FIG. 2 , theartifacts removal logic 218 comprises a plurality of functionalities220, 222, 224, 226 which are respectively trained and adopted to removeimage artifacts of a particular type from an image.

According to some embodiments, the image analysis system 200 isconfigured to generate, for each of the original images and for each ofthe artifact-generation-logics 230-236, a respective (single-fold)degraded image. The original images and the single-fold degraded imagesrelated to a particular artifact type are used for training anartifact-type specific artifacts removal logic. In some embodiments, theartifacts removal logic 218 is a single machine learning architecturewhich is trained on a training data set comprising degraded imagesrelating to many different artifact types and which comprisessub-functionalities (which may act as black boxes) respectively havinglearned to remove artifacts of a particular artifact type.

In other embodiments, each of the artifact-removal logics 220-226 isgenerated by training an untrained version of a machine learning logicon training data selectively comprising image artifacts of a particulartype. The generated, trained machine learning logic 220-226 are latercombined into a single software application which allows the user toremove artifacts of multiple different types from an input image.

According to preferred embodiments, the artifact-generation-logics230-236 are not only applied once on the original images for generatingsingle-fold degraded images 210.A. Rather, the image analysis system canbe configured for applying the artifact-generation-logics sequentially,whereby the degraded image output by any one of theartifact-generation-logics in the sequence is used as input by thefollowing artifact generation logics. For example, thestitching-artifact-generation-logic 230 can read an original image 208from the storage medium 240 and output a first intermediate imagecomprising a stitching artifact. The first intermediate image is thenused as input by the line-artifact-generation-logic 232 for generating asecond intermediate image which again is used as input for a furtherartifact-generation-logic.

In some embodiments, a user may specify the sequence of theartifact-generation-logics applied for generating multi-forwardsdegraded images 210.B manually.

Preferably, the user will specify the sequence of artifact generationoperations such that the sequence represents the “real” chronologicalsequence of artifact generation that is expected to occurred during thestaining and image acquisition. In other embodiments, the image analysissystem 200 automatically determines, for a given set or subset ofartifact-generation-logics 230-236, many or all of the combinatoriallypossible sequences of applying the way a artifact-generation-logics.Then, a training data set is generated that comprises the originalimages 208 and multi-fold degraded images 210.B, whereby at least someof the multi-fold degraded images have been created by the same set ofartifact-generation-logics but based on different chronologicalsequences. This may be beneficial, because the chronological sequence ofartifact generation may differ from case to case and a training data setthat comprises many different chronological sequences of applyingartifact types may allow the artifacts-removal logic 218 to “learn” manydifferent sequences of removing artifacts from images. This may bebeneficial, because depending on the particular staining protocol, thestaining dye, the stained tissue, and in the used image acquisitionsystem, different artifact types in different chronological sequencesmay be observed and the “real/most probable” sequence of artifactgeneration may not be known or may differ from case to case. Byautomatically generating highly information enriched training data set,embodiments of the invention may allow generating and artifacts removallogic that is able to accurately remove many different types ofartifacts in accordance with many different chronological artifactsremoval schemes from digital pathology images.

According to embodiments, each of the artifacts removal logics 220-226is implemented as a fully convolutional neural network. According toother embodiments, the combined artifacts removal logic 218 isimplemented as a single fully convolutional neural network.

In some embodiments, the machine-learning-logic that is employed forlearning the artifacts removal logic for individual or all artifacttypes is a neural network having a network architecture that isdescribed, for example in Long, Jonathan, Evan Shelhamer, and TrevorDarrell. “Fully convolutional networks for semantic segmentation.”Proceedings of the IEEE Conference on Computer Vision and PatternRecognition, 2015. Preferably, the network is a fully convolutionalnetwork trained end-to-end and pixels-to-pixels. The network can begenerated by adapting contemporary classification networks such asAlexNet, the VGG net, and GoogLeNet into fully convolutional networks.

The fully convolutional network is adapted for learning how to remove anartifact or a particular type by applying a model-basedartifacts-removal algorithm on a degraded version of an image forgenerating an automatically reconstructed image, comparing the originalimage with the reconstructed image and, in case the difference is toohigh (e.g. exceeds a threshold), modifying the model such that thedifference between the reconstructed image and the original image isminimized.

Thus, the machine learning logic “learns” a model that is capable ofremoving artifacts of a particular type from an image.

In some preferred embodiments, the machine learning logic 218, 220, 222,224, 226 is implemented as a “denoising autoencoder” with at least onetype of noise artifact model corresponding to the noise artifact typesdescribed herein. As a result, the trained machine-learning logic(“trained denoising encoder”) is adapted to remove artifacts of thisparticular type from an input image. The network architecture canconsist of a Deep Convolutional Neural Network setup with encoder anddecoder mechanisms e.g. FCN-8s (in Long, Jonathan, Evan Shelhamer, andTrevor Darrell. “Fully convolutional networks for semanticsegmentation.” Proceedings of the IEEE Conference on Computer Vision andPattern Recognition, 2015), or Lovedeep Gondara, Simon FraserUniversity, “Medical image denoising using convolutional denoisingautoencoders, ArXiv preprint arXiv:1608.04667, 2016. Another example fora suitable network architecture is presented in Unet (Ronneberger Olafet al.: “U-Net: Convolutional Networks for Biomedical ImageSegmentation”, chapter in “Medical Image Computing and Computer-AssistedIntervention”, MICCAI 2015: 18th International Conference, Munich,Germany, Oct. 5-9, 2015, Proceedings, Part III”, Springer InternationalPublishing”, ISBN=“978-3-319-24574-4”.

The training of the machine-learning logic 218, 220-226 comprisesinputting each artificially degraded image together with the respectiveoriginal non-degraded image to enable the autoencoder to learn torecreate the original image from the “degraded” image.

The artifact generation logics 230-236 are specified manually or aregenerated automatically or semi-automatically based on a machinelearning approach. For example, the stitching-artifact-generation-logiccan be obtained by automatically learning a square filter from one ormore images taken by a camera of a particular IAS of an empty slide. Tothe contrary, the salt-and-pepper noise generation function can bespecified explicitly by a programmer.

FIG. 3 is a digital tissue image depicting a “real” tissue foldartifact. This type of image artifact is often the result of the use ofblunt knifes during tissue cutting or of handling errors during theslide mounting. According to some embodiments of the invention, atissue-forward-artifact-generation logic is provided which is configuredto automatically add an artificial tissue-fold artifact on an inputimage for generating a degraded version of the image. The generation ofthe tissue-fold artifact comprises virtually cutting the original imageinto image parts, overlaying the image parts such that the parts overlayalong the cutting line, e.g. for an overlay margin of about 3 to 10pixels. The resulting degraded image looks very similar to an imagecomprising a “real” tissue fold artifact.

FIG. 4 is a digital tissue image with a “real” speckle noise artifact. Aspeckle noise is often caused by dust in the camera objects or on thecover slide of a tissue slide. Speckle noise can also be generated bybad pixels in the sensor cells of a camera. According to someembodiments, a speckle-artifact-generation-logic 234 implements asalt-and-pepper-noise generation function. This function is capable offaithfully reproducing the speckle noise often observed in digitalpathology images. Preferably, the function randomizes the radii of thespeckles.

FIG. 5 is a digital tissue image with a line artifact caused by an airbubble. According to some embodiments, a line-artifact-generation logic232 is configured to randomly select polynomial parameter of differentorders, to generate a black line in accordance with that parameters andwith the typical thickness of air bubble margins in a digital image.

FIG. 6 is a digital tissue image with a stitching artifact. A “stitchingartifact” or “stitching-noise-artifact” is an image artifact generatedby the optics or mechanics of the image acquisition system 204, wherebythe image acquisition system can in particular be a slide scanner.According to embodiments, a stitching-artifact-generation-logic 230implements a square filter that is applied on any input image receivedby the program logic 230, thereby generating and returning a degradedimage comprising a simulated stitching artifact. Preferably, the squarefilter is obtained empirically by analyzing the image of an empty slideobtained by a particular image acquisition system. The square filter isautomatically or manually fitted such that it faithfully reproduces theobserved stitching pattern of the image acquisition system. Then, in thestitching-artifacts-generation-logics 230 automatically generates, foreach of a plurality of original images, a respective degraded imagecomprising the simulated stitching artifact of a particular imageacquisition system. The original images and the degraded images are usedas training data set for training and stitching-artifact-removal logic220 that is capable of selectively removing the stitching artifactgenerated by a particular image acquisition system or type of imageacquisition system.

FIG. 7 is a digital tissue image with a “real” non-specific stainingartifact. Non-specific staining artifacts can be simulated, e.g. by aprogram logic 236, by applying a color-specific salt-and-pepper specklenoise generation function, whereby the color of this functioncorresponds to the type of stain whose unspecific staining artifactsshall be simulated and removed. For example, the program logic 226 ofthe image acquisition system is generated based on a machine learningapproach from a training data sets comprising original images 208 andartificially degraded images generated by program logic 236. The programlogic 226 is configured to selectively remove unspecific stainingartifacts of a particular stain from a digital pathology image.

FIG. 8 is a digital tissue image with a “real” bad-focus artifact thatis the result of a microscope focus loss. According to embodiments,degraded images comprising bad-focus noise can be automatically createdby modeling a point spread function (PSF) or a Gaussian approximation ofa PSF for a particular image acquisition system, e.g. for a particularmicroscope or microscope type.

FIG. 9 depicts a digital tissue image with a stitching artifact createdwith image acquisition system specific parameters;

FIG. 10 is a block diagram of a “naïve” Neural Network architecture usedas a machine learning logic according to embodiments of the invention.The depicted network architecture can be trained to learn the generationof an artifact of a single artifact type. Alternatively, the networkarchitecture can be trained to generate a predefined sequence ofartifacts of different types. In addition, or alternatively, the networkarchitecture depicted in this figure can be trained to remove anartifact of a particular artifact type or to remove multiple artifactsof different types in a predefined sequence. More layers can be insertedor removed to increase or decrease the complexity of the network. Byadding and removing layers from the network and by testing the accuracyof the trained network on a reference data set, a network architecturewith an optimum number of layers can be determined experimentally.

FIG. 11 is a block diagram of an “auxiliary loss” Neural Networkarchitecture used as machine learning logic in some other embodiments ofthe invention. The network architecture depicted in FIG. 11 canbasically be used in the same manner and for the same purposes as thearchitecture depicted in FIG. 10 .

Both the “naïve network” and the “auxiliary loss” network architecturecan be used as the program logic that learns to create and/or remove aparticular artifact type. Both the “naïve network” and the “auxiliaryloss” network architecture can be used as the program logic that learnsto create and/or remove sequences of artifacts. The “auxiliary loss”network, however, allows a tighter control of the training process,because sub-sets of layers in the network have assigned a “teacher” ontheir own which may allow to achieve a training of said sub-set oflayers to generate or remove a particular artifact type.

FIG. 12 a is a block diagram of a basic encoder unit as used, forexample, in a neural network depicted in FIG. 10 or 11 . The basicencoder unit forms a sequence of normalization layers wherein a batchnormalization layer precedes the rectification layer (rectified linearunit—ReLU) layer.

FIG. 12 b is a block diagram of a Residual Encoder Unit as used, forexample, in a neural network depicted in FIG. 10 or 11 . The basicencoder unit forms a sequence of normalization layers wherein the ReLUlayers precede the batch normalization layer.

FIG. 12 c is a block diagram of a Basic Decoder Unit that has a similararchitecture like the unit depicted in FIG. 12 a . In addition, itcomprises an unsampling layer.

Each Layer depicted in FIGS. 12 a-12 c has a function in a network, e.g.in one of the networks depicted in FIGS. 10 and 11 . For instance,convolotion2d applies 2d convolutional filters, batch normalizationnormalizes the gradients during training, upsampling makes the imagebigger, and pooling downsamples the image. The layers can respectivelycomprise parameters whose values are learned (optimized) during thetraining phase.

The invention claimed is:
 1. A method for providing program logicadapted to remove artifacts from digital tissue images, the methodcomprising: creating first image-artifact-generation logic, the firstimage-artifact-generation logic being configured for specificallygenerating a first type of artifact, the first image-artifact-generationlogic being A) an image-acquisition-system-specificimage-artifact-generation logic or B) atissue-staining-artifact-generation logic; generating, for each of aplurality of original images respectively depicting a tissue sample, afirst artificially degraded image by applying the firstimage-artifact-generation logic on each of the original images; andgenerating a program logic configured for removing artifacts fromdigital tissue images, the generating the program logic includingtraining an untrained version of a first machine-learning logic thatencodes a first artifacts-removal logic on the original images and theirrespectively generated first degraded images, and returning the trainedfirst machine-learning logic as the program logic or as a componentthereof, wherein the first image-artifact-generation logic is theimage-acquisition-system-generation logic, the firstimage-artifact-generation logic is a bad-focus-artifact-generationlogic, and wherein the creating the first image-artifact-generationlogic includes: adding beads of known size on a slide, taking, by theimage acquisition system that is used for capturing the plurality oforiginal images, a digital image of the slide with the beads, analyzingsaid digital image for automatically determining an optical pattern andits size generated by at least one of the beads, identifying a functionadapted to simulate the generation of the pattern of the determined sizefrom a spot having the known bead size, and creating the firstimage-artifact-generation logic by integrating the identified functioninto a machine-executable code.
 2. The method of claim 1, the functionadapted to simulate the generation of the pattern being a point spreadfunction or a Gaussian filter.
 3. The method of claim 1, wherein thefirst image-artifact-generation logic is theimage-acquisition-system-generation logic, and further comprising:providing a plurality of image acquisition systems and for each of theplurality of image acquisition systems: creating the firstimage-artifact-generation logic according to claim 1, the firstimage-artifact-generation logic being specific to said image acquisitionsystem; capturing, by said image acquisition system, a plurality oforiginal images of a tissue sample; generating, for each of theplurality of original images a respective first artificially degradedimage by applying the created image-acquisition-specific firstimage-artifact-generation logic on each of the original images;generating an image-acquisition-system-specific program logic configuredfor removing artifacts from digital tissue images captured by said imageacquisition system, the generation comprising training an untrainedversion of a first machine-learning logic that encodes a firstartifacts-removal logic on the original images and their respectivelygenerated first degraded images; and returning the trained firstmachine-learning logic as the image-acquisition-system-specific programlogic or as a component thereof.
 4. The method of claim 1, furthercomprising: generating, for each of the original images, a secondartificially degraded image by applying a secondimage-artifact-generation logic on each of the original images, thesecond image-artifact-generation logic being configured for specificallygenerating a second type of artifact; the generation of the programlogic configured for removing artifacts further comprising: training anuntrained version of a second machine-learning logic that encodes asecond artifacts-removal logic on the original images and theirrespectively generated second degraded images; and combining the trainedfirst machine-learning logic with the trained second machine-learninglogic for providing the program logic, the program logic beingconfigured for removing artifacts of at least the first and the secondartifact type.
 5. The method of claim 4, wherein the secondimage-artifact-generation logic is theimage-acquisition-system-generation logic, the secondimage-artifact-generation logic is astitching-artifact-generation-logic, and the method further comprisescreating the second image-artifact-generation logic by: taking, by theimage acquisition system that is used for capturing the plurality oforiginal images, a digital image of an empty slide; analyzing thedigital image of the empty slide for automatically creating a squarefilter that is configured to simulate a stitching artifact generated bythe image acquisition system; and creating the secondimage-artifact-generation logic by integrating the square filter into amachine-executable code.
 6. The method of claim 4, wherein the secondimage-artifact-generation logic is theimage-acquisition-system-generation logic, the secondimage-artifact-generation logic is a hue-shift artifact-generation-logicconfigured to specifically generate hue shift artifacts that aregenerated by the image acquisition system, the method further comprisescreating the second image-artifact-generation logic by: taking, by theimage acquisition system that is used for capturing the plurality oforiginal images, a digital image of an empty slide; analyzing thedigital image of the empty slide for automatically creating a hue shiftfilter that is configured to simulate the hue-shift artifact generatedby the image acquisition system; and creating the secondimage-artifact-generation logic by integrating the hue shift filter intoa machine-executable code.
 7. The method of claim 4, wherein the secondimage-artifact-generation logic is thetissue-staining-artifact-generation logic, the secondimage-artifact-generation logic is atissue-fold-artifact-generation-logic configured to generatetissue-fold-artifacts in an input image, and the method furthercomprises creating the second image-artifact-generation logic by:generating at least two image parts of each of one or more input imagesby cutting each input image across a randomly selected line; overlayingthe at least two image parts along the line such that an overlap of thetwo image parts along the line of the cut is created; and merging theoverlap of the image parts, thereby generating a degraded version of theinput image; training a machine learning logic on the one or more inputimages and the degraded image for providing a trained machine learninglogic adapted to simulate the merged, degraded image from the inputimage that was split for generating the merged, degraded image; andcreating the second image-artifact-generation logic by integrating saidtrained machine learning logic into a machine-executable code.
 8. Themethod of claim 4, wherein the second image-artifact-generation logic isthe tissue-staining-artifact-generation logic, the secondimage-artifact-generation logic is a line-artifact-generation-logicadapted to specifically generate line-artifacts in an input image, thelength, thickness and/or curvature of the line-artifacts representinghair, gas bubble outlines and/or cloth fibers, and the method furthercomprises creating the second image-artifact-generation logic by:generating a plurality of polynomial curves using randomly generatedvalues as curve parameters; inserting black lines along the coordinatesof the polynomial curves in one or more input images, thereby generatinga degraded version of the input image; training a machine learning logicon the one or more input images and the degraded image generatedtherefrom for providing a trained machine learning logic adapted tosimulate the line artifacts; and creating the secondimage-artifact-generation logic by integrating said trained machinelearning logic into a machine-executable code.
 9. The method of claim 4,wherein the second image-artifact-generation logic is thetissue-staining-artifact-generation logic, the secondimage-artifact-generation logic is a color-noise-generation-logicadapted to specifically generate artifacts consisting of optical noiseof a defined color or color combination in an input image, and themethod further comprises creating the second image-artifact-generationlogic by: specifying a color-specific salt-and-pepper function adaptedto simulate tissue slides regions with residual stain; and creating thesecond image-artifact-generation logic by integrating saidcolor-specific salt-and-pepper function into a machine-executable code.10. The method of claim 4, the second image-artifact-generation logicbeing A) an image-acquisition-system-specific image-artifact-generationlogic or B) a tissue-staining-artifact-generation logic, the secondimage artifact generation logic being different from the firstimage-artifact-generation logic.
 11. The method of claim 1, the programlogic configured for removing artifacts being configured for removingartifacts of multiple different types, the method further comprising:providing, for each of the different artifacts types, a respectiveartifact-generation logic configured for specifically generating saidparticular type of artifact; applying, according to a first sequence,the different image-artifact-generation logics on each of the originalimages, whereby the image-artifact-generation logic at the firstposition within the first sequence takes each of the original images asinput and wherein the degraded image output by saidimage-artifact-generation logic and any image-artifact-generation logicat a subsequent position within the first sequence is used as inputimage by the next one of the different image-artifact-generation logicsin the first sequence, thereby generating, for each of the originalimages, a first multi-fold degraded image; the generation of the programlogic further comprising: training a machine-learning logic that encodesa multi-step-artifacts-removal logic, the training being performed on atleast the original images and their respectively generated firstmulti-fold degraded images; and using said trainedmulti-artifact-machine-learning logic as the program logic or as part ofthe program logic, the program logic being configured for sequentiallyremoving artifacts of each of the multiple types of artifacts.
 12. Themethod of claim 11, further comprising: applying, according to at leasta second sequence, different image-artifact-generation logics on theoriginal images, whereby the image-artifact-generation logic at thefirst position within in the second sequence takes each of the originalimages as input and wherein the degraded image output by saidimage-artifact-generation logic and any image-artifact-generation logicat a subsequent position within the second sequence is used as inputimage by the next one of the different image-artifact-generation logicsin the second sequence, thereby generating, for each of the originalimages, a second multi-fold degraded image; wherein the training of theuntrained version of the machine-learning logic is performed on at leastthe original images and their respectively generated first and secondmulti-fold degraded images.
 13. The method of claim 1, furthercomprising: automatically creating a plurality of sequences of thedifferent image-artifact-generation logics, each of the plurality ofsequences comprising different image-artifact-generation logics at afirst, a second and at least a third position by permutating thepositions of the image-artifact-generation logics within the respectivesequence; and applying, for each of the plurality of sequences, theimage-artifact-generation logics of said sequence on the originalimages, whereby the image-artifact-generation logic at the firstposition within in said sequence takes each of the original images asinput and wherein the degraded image output by saidimage-artifact-generation logic and any image-artifact-generation logicat a subsequent position in said sequence is used as input image by thenext one of the different image-artifact-generation logics in saidsequence, thereby generating, for each of the original images and foreach of the plurality of sequences, a multi-fold degraded image, whereinthe training of the untrained version of the machine-learning logic isperformed on at least the original images, the plurality of sequencesand their respectively generated multi-fold degraded images.
 14. Themethod of claim 11, the first sequence being defined manually andrepresenting a known sequence of artifact generation in a tissuestaining and image acquisition workflow.
 15. The method of claim 1,wherein the machine-learning logic trained for providing theimage-artifact-generation logic and/or for providing the program logicconfigured for removing artifacts is a neural network.
 16. The method ofclaim 1, wherein the machine-learning logic trained for providing theimage-artifact-generation logic and/or for providing the program logicconfigured for removing artifacts is an autoencoder.
 17. Animage-correction method for tissue images depicting a biological sample,the method comprising: receiving a digital image of the biologicalsample, the digital image comprising an artifact; applying the programlogic generated in accordance with claim 1 on the received digital imagefor generating an artifact-corrected image; and returning theartifact-corrected image.
 18. A non-transitory computer-readable mediumsoring computer-interpretable instructions which, when executed by aprocessor, cause the processor to perform the method of claim
 1. 19. Animage analysis system comprising: a storage medium comprising aplurality of original images, each original image depicting a tissue; aprocessor configured to: create first image-artifact-generation logic,the first image-artifact-generation logic being configured forspecifically generating a first type of artifact, the firstimage-artifact-generation logic being A) animage-acquisition-system-specific image-artifact-generation logic or B)a tissue-staining-artifact-generation logic, generate, for each of theoriginal images, a first artificially degraded image by applying a firstimage-artifact-generation logic on each of the original images; generatea program logic configured for removing artifacts from digital tissueimages, the generating the program logic including training an untrainedversion of a first machine-learning logic that encodes a firstartifacts-removal logic on the original images and their respectivelygenerated first degraded images, and returning the trained firstmachine-learning logic as the program logic or as a component thereof,wherein the first image-artifact-generation logic is theimage-acquisition-system-generation logic, the firstimage-artifact-generation logic is a bad-focus-artifact-generationlogic, and wherein the processor is configured to create the firstimage-artifact-generation logic by: adding beads of known size on aslide, taking, by the image acquisition system that is used forcapturing the plurality of original images, a digital image of the slidewith the beads, analyzing said digital image for automaticallydetermining an optical pattern and its size generated by at least one ofthe beads, identifying a function adapted to simulate the generation ofthe pattern of the determined size from a spot having the known beadsize, and creating the first image-artifact-generation logic byintegrating the identified function into a machine-executable code. 20.A method for providing program logic adapted to remove artifacts fromdigital tissue images, the method comprising: creating firstimage-artifact-generation logic, the first image-artifact-generationlogic being configured for specifically generating a first type ofartifact, the first image-artifact-generation logic being A) animage-acquisition-system-specific image-artifact-generation logic or B)a tissue-staining-artifact-generation logic; generating, for each of aplurality of original images respectively depicting a tissue sample, afirst artificially degraded image by applying the firstimage-artifact-generation logic on each of the original images; andgenerating a program logic configured for removing artifacts fromdigital tissue images, the generation the program logic includingtraining an untrained version of a first machine-learning logic thatencodes a first artifacts-removal logic on the original images and theirrespectively generated first degraded images, and returning the trainedfirst machine-learning logic as the program logic or as a componentthereof, wherein the first-image-artifact-generation logic is thetissue-staining-artifact-generation logic, and wherein the firstimage-artifact-generation logic is one of (i) atissue-fold-artifact-generation-logic configured to generatetissue-fold-artifacts in an input image or (ii) aline-artifact-generation-logic adapted to specifically generateline-artifacts in an input image, the length, thickness and/or curvatureof the line-artifacts representing hair, gas bubble outlines and/orcloth fibers, the method further comprising creating the firstimage-artifact-generation logic.
 21. The method of claim 20, whereinwhen the first image-artifact-generation logic is thetissue-fold-artifact-generation-logic, the creating the firstimage-artifact-generation logic comprises: generating at least two imageparts of each of one or more input images by cutting each input imageacross a randomly selected line, overlaying the at least two image partsalong the line such that an overlap of the two image parts along theline of the cut is created, merging the overlap of the image parts,thereby generating a degraded version of the input image, training amachine learning logic on the one or more input images and the degradedimage for providing a trained machine learning logic adapted to simulatethe merged, degraded image from the input image that was split forgenerating the merged, degraded image, and creating the firstimage-artifact-generation logic by integrating said trained machinelearning logic into a machine-executable code; and when the firstimage-artifact-generation logic is the line-artifact-generation-logic,the creating the first image-artifact-generation logic comprises:generating a plurality of polynomial curves using randomly generatedvalues as curve parameters, inserting black lines along the coordinatesof the polynomial curves in one or more input images, thereby generatinga degraded version of the input image, training a machine learning logicon the one or more input images and the degraded image generatedtherefrom for providing a trained machine learning logic adapted tosimulate the line artifacts, and creating the firstimage-artifact-generation logic by integrating said trained machinelearning logic into a machine-executable code.